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placement in jobs related to students' program of study, community economic 
return, course utilization, retention/graduation rate, and faculty 
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attached. (HAA) 
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DRAFT 

NTC INSTRUCTIONAL TEAM LEADERSHIP COUNCIL 



PROGRAM ASSESSMENT CRITERIA SUMMARY 

Mission Statement: Northcentral Technical College provides quality education, training, and 
services of high value to individuals, businesses, industries, and organizations-building a 
competitive work force in a changing global society. 

The Program Assessment Criteria includes both measurements and traits . Sources for 
measurements are from commonly used (and audited) college or state data, such as six-month 
follow-up, staff accounting, client reporting system, course files, etc. Traits are measurable but not 
necessarily from readily available and audited sources. Trait information is provided from program 
faculty and describes such things as employer satisfaction and the supply and demand for the 
program. 

I. Customer Satisfaction can be assessed by: 

1. MEASUREMENT: "Student Satisfaction Rate" 

As reported in the 6-month follow-up for respondents who reported very satisfied or 
satisfied with their program. The data is presented for the prior three years and as a 
three-year average. 

BENCHMARK: Three-year average greater than 95%. 

2. MEASUREMENT: "Student Satisfaction Rate including Leavers" 

A similar report will be developed for program leavers to be implemented fall 
semester 1996 for fall and spring 1995-96 leavers. 

BENCHMARK: To be determined. 

3. TRAIT: "Employer Satisfaction" 

Exemplary programs are regularly involved with related business and industry to 
obtain feedback on employer satisfaction of programs. 

BENCHMARK: Program can produce evidence of above. 

Provide evidence that the program faculty is regularly involved with business and 
industry to obtain feedback on employer satisfaction with the program and program 
graduates. This could include work with advisory committees but broader exposure 
is desirable. 
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II. Economic Impact can be assessed by: 



1. MEASUREMENT: "Graduate Salary Comparison" 

NTC graduate hourly average salaries compared to those reported in WTCS 6-month 
graduate follow-up expressed as a percentage. The three prior years are reported 
and a three-year average. 

BENCHMARK: 100% of state average 

2. MEASUREMENT: "Placement" 

The percent of placement into related employment by program as reported in the NTC 
6-month follow-up survey. The data is presented for the prior three years and as a 
three-year average. 

BENCHMARK: 90% 

3. MEASUREMENT: "Placement with Leavers" 

The percent of placement into related employment including information provided by 
leavers by program. The data is presented for the prior three years and as a three- 
year average. 

BENCHMARK: To be determined. 

4. TRAIT: "Future Demand" 

An exemplary program finds ways to assess labor market needs, supply and demand, 
future trends and political implications. 

BENCHMARK: Program can produce evidence of above 

A. Provide data on labor market needs 

Source: DILHR - State and regional (with central District assistance) 

B. Provide supply data 

Source: WTCS, client reporting system, graduates and enrollments 
(with central District assistance) 

Source: Other training sources (include high schools, if appropriate) 

C. Provide data on future emplacement trends 

Source: DILHR job trends (with central District assistance) 

Source: Professional publications 
Source: Other 

D. Describe the current political and economic environment as it may relate to major 
program modification or discontinuance 
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E. Provide out-of-district enrollments 
Source: WTCS report 

NOTE: The above information could be used by the program and advisory committee 
for program improvement and budget requests. 

5. MEASUREMENT: "Community Economic Return" 

As based on the annual salary of employed related graduates divided by direct 
instructional cost. 

Annual salary is determined by information from the 6-month follow-up survey (job- 
outs to be added later). Annual salary is computed by prorating hours worked for 
all respondents. 

Direct instructional costs include salaries and benefits of full, part-time and call staff" 
for all program courses. Course costs are based on staff accounting information and 
are prorated by program students enrolled. This does include elective courses taken 
by program students. It does not include other costs directly attached to an 
instructional area, capital equipment or general overhead. 

Emploved-related Graduates x Annual Salary = Community 

Direct Instructional Cost Economic Return 

BENCHMARK: Variable by program type (TBD) 

6. TRAIT: "5 Year Follow-up" 

An Exemplary Program tracks graduates beyond the 6 month follow-up and uses 
graduate information to improve the program. 

BENCHMARK: The program can provide evidence of tracking 
graduates past the 6-month follow-up. 

Provide information on your program students success in the labor market 
beyond the 6 month follow-up. Include salary and job advancement and retention 
information. 

Source: 5 year follow-up and other data. Note: The college is exploring 
the possibility to annually survey our graduates for three years after graduation. 
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III. Efficient Use of Resources assessed by: 



1. MEASUREMENT: "Course Utilization" 

As a percent of occupational specific courses by calculating actual hours of 
instruction (AHI) and dividing by the capacity hours of instruction (CHI). 

AHI = actual headcount as reported to the state X total course hours as defined by 
course section 

CHI = capacity as defined by the course section X total course hours as defined by 
the course section 

BENCHMARK: 65% utilization 

2. TRAIT: "Student Accessibility" 

An exemplary program identifies and responds to student accessibility needs in 
educationally sound ways that are in sync with the market served, utilizing cost 
effective solutions. 

BENCHMARK: Annually complete Program Accessibility Matrix 

3. MEASUREMENT: "Retention/Graduation Rate" 

(%) Based on the number of students who started a program in a given year 
compared to successful student outcome. Successful outcomes include graduations, 
still in program, job-outs (in the future). Time duration covers students officially in a 
program (not pre-program students) allowing one and one half (1.5) the time for 
normal program completion. Short term (Ex: a 6-week Nursing Assistant) completion 
is within a year. Long-term will be considered the same as two-year programs. 

BENCHMARK: Retention rate greater than 70%. 

4. MEASUREMENT: "Faculty Utilization" 

Based on actual hours of instruction (AHI) per FTE faculty for all program courses 
(not electives). The data is presented for the prior three years and as a three-year 
average. 

.AHI = actual headcount as reported to the state multiplied by the total course hours 
as defined by course section 

FTE Faculty = staff accounting instructor loads prorated by the number of program 
students in the course section 

BENCHMARK: Three-year average greater than 12,000 hours 
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5. TRAIT: "High Student Demand" 

An exemplary program has a high student demand as evidenced by a full program. 
BENCHMARK: Program is full at start-up 

A. Describe the student demand for your program over the last three years 
Source: NTC admissions reports 

B. Describe your student market in terms of student demographics, age, enrollment 
status, location, prior education, entry-level competencies, etc. 

Source: Client Reporting System 

C. If you have a non-homogeneous group, describe your student market segments by 
group. NOTE: This information will be helpful to develop a marketing plan, if 

necessary. 
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TUDENT SATISFACTION RATE 



TOTALS 






















1.1 Satisfai 


;tion 








1992-199 


1993-199 


1994-199 


Average 


Accounting 


100.00% 


95.40% 


97.80% 


97.73% 


Marketing 


85.70% 


96.40% 


94.70% 


92.27% 


Legal Secretary 


100.00% 


94.70% 


94.70% 


96.47% 


Medical Secretary 


100.00% 


100.00% 


100.00% 


100.00% 


Admin. Assistant 


95.00% 


100.00% 


100.00% 


98.33% 


CIS Programmer 


95.20% 


85.00% 


88.80% 


89.67% 


Small Busi Mgt. 


100.00% 


100.00% 


100.00% 


100.00% 


Insurance Services 


100.00% 


100.00% 


100.00% 


100.00% 


Supervisory Mgt. 


100.00% 


100.00% 


100.00% 


100.00% 


Printing/Publishing 




92.30% 


90.00% 


91.15% 


Police Science 


93.70% 


90.00% 


94.10% 


92.60% 


Dental Hygiene 


93.70% 


100.00% 


95.40% 


96.37% 


Nursing AD 


80.70% 


90.00% 


100.00% 


90.23% 


Radiography 


100.00% 


100.00% 


87.50% 


95.83% 


Edu Interpreter Tec 


93.30% 


100.00% 


91.60% 


94.97% 


Electronics 


100.00% 


100.00% 


100.00% 


100.00% 


Mechanical Design 


100.00% 


100.00% 


100.00% 


100.00% 


Arch-Res Design 


90.40% 


100.00% 


90.90% 


93.77% 


Electromechanical 


100.00% 


100.00% 


95.60% 


98.53% 


Laser Tecnology 


100.00% 


100.00% 


100.00% 


100.00% 


Industrial Engineer 


100.00% 


100.00% 


100.00% 


100.00% 


Farm Business 




87.50% 


100.00% 


93.75% 


Police Recruit 


100.00% 


97.30% 


91.60% 


96.30% 


Nursing Assistant 


95.80% 


99.10% 


89.60% 


94.83% 


EMT 


100.00% 


100.00% 


91.80% 


97.27% 


IPS 


100.00% 


100.00% 


100.00% 


100.00% 


Inventory Control 


100.00% 


50.00% 


91.60% 


80.53% 


Auto Body/Paint 


100.00% 


100.00% 




100.00% 


Mechanical/Comput 


100.00% 


100.00% 


100.00% 


100.00% 


Welding 


92.80% 


100.00% 


94.40% 


95.73% 


Surgical Tech. 


100.00% 


85.70% 


100.00% 


95.23% 


Agri Power & Equip. 


100.00% 


100.00% 


66.60% 


88.87% 


Automotive Tech. 


100.00% 


90.00% 


100.00% 


96.67% 


Machine Tooling 


50.00% 


90.90% 


100.00% 


80.30% 






















TOTALS/AVGS 


95.82% 


95.42% 


95.35% 


95.51% 












Measurement Goal 
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GRADUATE SALARY COMPARISON 
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60.50% 


66.40% 


64.80% 


63.90% 


Marketing 


68.00% 


71.00% 


79.40% 


72.80% 


Legal Secretary 


52.80% 


58.80% 


50.40% 


54.00% 


Medical Secretary 


55.30% 


67.80% 


55.60% 


59.57% 


Admin. Assistant 


50.90% 


57.80% 


47.30% 


52.00% 


CIS Programmer 


48.70% 


58.80% 


77.30% 


61.60% 


Small Busi Mgt. 


48.00% 


64.30% 


69.60% 


60.63% 


Insurance Services 


64.60% 


65.20% 


65.80% 


65.20% 


Supervisory Mgt 


65.00% 


81.10% 


62.40% 


69.50% 
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84.30% 


70.50% 


59.00% 
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82.50% 
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62.60% 
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62.00% 


63.40% 
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72.00% 
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Electromechanical 


61.90% 


58.80% 


64.50% 
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58.30% 
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61.97% 
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67.20% 
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80.40% 
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Agri Power & Equip. 


30.70% 


29.50% 
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26.07% 


Automotive Tech. 


67.50% 


96.40% 


72.70% 


78.87% 


Machine Tooling 


52.90% 


77.20% 


80.70% 


70.27% 






















TOTALS/AVGS^ 


61.59% 


66.83% 


63.74% 


64.05% 




















111.1 Course 


l Utilization 


(65%) 














RETENTION/GRADUATION RATE 



ERIC 




f NWtWO 






cono^fMn^u) 



os 



oo 

7*-a:;j 



IX 



BEST COPY AVAIIABI 










(OQ9SO)(n^b)N) 



FACULTY UTILIZATION 



1 



10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 

43 

44 

45 



1 










I 










[ 
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1 


1992-199 


1993-199 


1994-199 i 


Average 


Accounting 


15640 


16518 


15258 


15805 


Marketing 


15364 


14733 


14578 


14892 


Legal Secretary 


14683 


15071 


14890 


14881 


Medical Secretary 


15000 


17829 


14592 


15807 


Admin. Assistant 


13556 


15161 


12981 


13899 


CIS Programmer 


13751 


13619 


12553 


13308 


Small Busi Mgt. 


12835 


13740 


12735 


13103 


Insurance Services 


12756 


13528 


10351 


12212 


Supervisory Mgt. 


10684 


16675 


11419 


12926 


Printing/Publishing 


11748 


11312 


10348 


11136 


Police Science 


17311 


17176 


15373 


16620 


Dental Hygiene 


8434 


8651 


9770 


8952 


Nursing AD 


9796 


9735 


9077 


9536 


Radiography 


14776 


14576 


11583 


13645 


Edu Interpreter Tec 


10615 


9826 


10781 


10407 


Electronics 


11752 


13223 


10572 


11849 


Mechanical Design 


9763 


10503 


11621 


10629 


Arch-Res Design 


14762 


12809 


14201 


13924 


Electromechanical 


11814 


12505 


11807 


12042 


Laser Tecnology 


11082 


12193 


10728 


11334 


Industrial Engineer 


12494 


2847 


8386i 


7909 


Farm Business 


13586 


12103 


12288 


12659 


Police Recruit 


15205 


14241 


11973 


13806 


Nursing Assistant 


16726 


10473 


13050 


13416 


EMT 


11434 


11681 


14743 


12619 


IPS 


14697 


14204 


9764 


12888 


Inventory Control 


342” 


10392 


13921 


8218 


Auto Bodv/Paint 


8641 


11714 


12019 


10791 


Mechanical/Comput 


8815 


8786 


9072 


8891 


Welding 


10383 


12686 


11155 


114Q8. 


Surgical Tech. 


8301 


8442 


6826 


7856 


Agri Power & Equip 


7620 


10479 


6753 


8284 


Automotive Tech. 


14585 


15419 


12031 


14012 


Machine Tooling 


10231 


12784 


12874 


11963 






















TOTALS/AVGS 


13089 


1 12877 


12121 


12696 
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DESIGNING AND 

Implementing value 

FOCUSED EFFECTIVENESS 
INDICATORS 

JUNE 23,1997 / TEMPE, A Z 
BECKY RAJEK, ARVES JONES, 
DAN McCOLLUM, TOM GOLTZ 
NORTHCENTRAL TECHNICAL 
COLLEGE, WAUSAU, WISCONSIN 



WHY VALUE FOCUS? 

1LTAXPAYERMQQD 

PRISONS 

WELFARE REFORM 
LOW TAXES 
ACCOUNTABILITY 

11 QUALITY AT A REASONABLE COST 
Hi BETTER, FASTER, LOW COST PROVIDER 
il EDUCATION IS A MATURE INDUSTRY 



WHY VALUE FOCUS? (page 2) 




NTC MISSION 


t> CARVER POLICY GOVERNANCE MODEL 
% WHAT SERVICE 
« TO WHOM 
* AT WHAT COST 

STATE MANDATES FOR COST CONTROL 
CUSTOMER FOCUS 
NTC MISSION 

NCA - DOING WHAT WE SAY WE ARE 
DOING 


Northcentra! Technical College provides quality 
education, training, and services of high value 
to individuals, business and industry, and 
organizations, building a competitive work 
force in a changing global society. 















NTC VISION 



NTC is the college of choice, anticipating and 
exceeding customers' expectations with 
excellence 



WHO ARE THEY? 
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WHAT DO THEY BUY? 




/SAOMSii '/"I'?" :: 
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GUIDELINES FOR DEVELOPMENT OF 
PROGRAM EFFECTIVENESS 
INDICATORS 




PURPOSE AND OBJECTIVES 


m MUST BE OF HIGH QUALITY 
m MUST BE OF HIGH VALUE 
m MUST BUILD A COMPETITIVE 
WORKFORCE 
IS CUSTOMER DRIVEN 
m MUST BE ADAPTABLE 
a MEASURABLE AND MANAGEABLE 


m DEVELOP CLEAR, MEASURABLE 
STANDARDS 

M ELIMINATE THE "RANKING" OF 
PROGRAMS 

m PROVIDE CLEAR DIRECTION TO 
FACULTY AND STAFF 
a EMPOWER FACULTY TO PROACTIVELY 
WORK TO IMPROVE THEIR PROGRAMS 


• * * ^ ^ :::x: :: x ; ; :• w *•£££: 




" .. ' ... - 

<:::v :■ .. • 









DEVELOPMENT 




MEASURES 


a STRONG FACULTY INVOLVEMENT 
il BOARD GOAL 




a STUDENT SATISFACTION RATE 
« STUDENT SATISFACTION RATE 
INCLUDING "LEAVERS" 






it GRADUATE SALARY COMPARISON 
a RELATED PLACEMENT 
m RELATED PLACEMENT WITH "LEAVERS" 

If COMMUNITY ECONOMIC RETURN 
a COURSE UTILIZATION 

a RETENTION/GRADUATION RATE^^*****^*-- 


g mmtm • • " = 

' y-:;-:. ;••• , ; ' \ :;V V > X* V.+.-.-'J . '. : ! : : 




m FACULTY 


v.wav/.vav.%v/.%v/.%v.v.v.wmvavas , .ViV.v/.%v/>.v!n>v//,v!v.v!%v;‘;vXvX%%%%%vX^ !. .; . . , , . Mv ,'.vX* ....... 







TRAITS 




IMPLEMENTATION 


M EMPLOYER SATISFACTION 




■ MEASURES DISTRIBUTED 


M FUTURE DEMAND 




W TRAINING PROVIDED 


M 5 YEAR FOLLOW-UP 




M FLEXIBLE FEEDBACK 


M STUDENT ACCESSIBILITY 






ffl HIGH STUDENT DEMAND 






wMmmmmmmMmmmmmMmmM worn 













USES 




VALUE FOCUS 


II PROGRAM SCREENING 




11 VALUE IS THE ESTIMATE OF WORTH 


m IDENTIFY PROGRAMS NEEDING 




EXPRESSED IN MONEY 


IMPROVEMENT 




li VALUE equals BENEFIT divided by COST 


m BUILD FACULTY UNDERSTANDING OF 




m VALUE IS CUSTOMER FOCUSED 


COSTAND BENEFIT 




W VALUE IS CUSTOMER DEFINED 


li BECAME BASE TO IMPROVE PROGRAM 
MIX 






mmmmm ' v\\ . ^ ' » * 

s ”, " ’ / * : v . * x • • • : . • v . . ::: • :: : . ■ /. : ; • • : . : : 
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NTC VALUE FORMULA - COMMUNITY 
ECONOMIC RETURN 



m VALUE EQUALS THE TOTAL SALARY OF 
EMPLOYED RELATED GRADUATES 
DIVIDED BY THE DIRECT 
INSTRUCTIONAL COST 



THE POWER OF THE VALUE RATIO 




THE POWER OF VALUE 


m SELECT THE BENEFIT THAT IS 




m ALLOWS STAFF TO SELECT THE MOST 


IMPORTANT TO THE CUSTOMER AND 




IMPORTANT AREAS TO IMPROVE 


THE MISSION OF THE COLLEGE 




► REDUCE COST 


m POSITIVELY COMBINES SEVERAL 




► INCREASE RECRUITMENT 


TRADITIONAL INDICATORS 




► INCREASE RETENTION 


li THE HIGHER THE RATIO THE MORE 




► IMPROVE WORTH OF GRADUATES 


POSITIVE THE BENEFIT TO THE 
CUSTOMER 




► ETC. 


m EXCELLENT TOOL TO IMPROVE 






PROGRAM MIX! 7 
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THE POWER OF VALUE 



m USES MEASURABLE DATA BUT 
FOCUSES ON THE FUTURE 
ANTICIPATE THE NEEDS OF THE 
CUSTOMER 

EXCEED THE NEEDS OF THE CUSTOMER 
* LOW COST PROVIDER 
JUST IN TIME DELIVERY 
ENCOURAGES PARTNERSHIPS 
M FLEXIBLE 



EXAMPLES OF PROGRAM 
IMPROVEMENT 



m RESULTS EXCEEDED EXPECTATIONS 
EITP CASE STUDY 

INFORMATION PROCESSING SPECIALIST 
REDESIGNS CURRICULUM 
^ AUTOMOTIVE TECHNICIAN PROGRAM 
EXPLORING NEW PARTNERSHIPS 



OUTCOMES 




OPPORTUNITIES FOR CHANGE 


m EACH PROGRAM RECEIVED 
CONSTRUCTIVE FEEDBACK 
M NINE PROGRAMS IDENTIFIED 
11 STATE MODEL 
H EMPOWERMENT HAPPENS 


m PROCESS NEEDS TO RECOGNIZE 
PROGRAMS THAT WORK WITH SPECIAL 
POPULATIONS 

m PROCESS DOES NOT ADEQUATELY 
RECOGNIZE WORK IN PROGRESS 
11 MEASUREMENTS OF CUSTOMER 
SATISFACTION NEED TO BE 
STRENGTHENED 

M PROGRAMS NEED TO BE PROVIDED WITH 
ADDITIONAL ACCESSJQ ENVIRONMENTAL 
SCANNING AND CABQR FORCE DATA-' 
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